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Abstract 

We derive the second-order sampling properties of certain autocovariance and autocorrelation esti- 
mators for sequences of independent and identically distributed samples. Specifically, the estimators we 
consider are the classic lag windowed correlogram, the correlogram with subtracted sample mean, and 
the fixed-length summation correlogram. For each correlogram we derive explicit formulas for the bias, 
covariance, mean square error and consistency for generalised higher-order white noise sequences. In 
particular, this class of sequences may have non-zero means, be complexed valued and also includes 
non-analytical noise signals. We find that these commonly used correlograms exhibit lag dependent 
covariance despite the fact that these processes are white and hence by definition do not depend on lag. 

1 Introduction 

Serial correlation, although not as popular as its Wiener-Khintchine theorem equivalent, spectral analysis, 
is still of fundamental importance in the analysis of time series, and is used in a wide range of applications 
ranging from radio astronomy 1 1 2 1 and radar scatter from random media 1 1 1 1 to DNA sequencing 1 4 1 and 
wave-particle interaction instruments for space plasma research (S). 

One of the most fundamental problems in time series analysis is to discern if the samples in a given 
series are independent. Under stationary conditions, this problem is most naturally dealt with using the 
autocorrelation sequence (ACS) of the series since it measures the correlation between samples in the 
series which are separated by some interval of time known as the lag. In practice, we can only estimate 
an ACS based on a finite number of data samples. So if we are to successfully detect the independence 
of the samples we must know the sampling errors of the ACS estimator for independent and identically 
distributed (IID) sample sequences, which are by definition independent. For our purposes of deriving the 
sampling errors it turns out that we can in fact broaden slightly the class of sequences from IID sequences 
to higher-order white noise or, as we will sometimes call it, general white noise. 

It is of course the sampling errors or sampling properties that limit how well we can determine sample 
independence from the estimated ACS and it is therefore an important issue to explicitly provide them for 
general white noise sequences. General sampling properties of ACSs were first studied in \2\. Despite 
considerable subsequent attention, it seems the important special case of general IID sample sequences has 
not been fully exhausted. For instance, the sampling properties of autocovariance estimates of complex 
valued noise has not been published and neither has the case of autocorrelation estimates of nonzero mean 
noise series. To be clear, the difference between what we call autocorrelation and autocovariance is that in 
autocovariance the mean is explicitly subtracted from data samples while in autocorrelation it is not. Our 
use of these terms conforms with 1 3 1. We will use the term correlogram as a collective term for estimators 
of autocovariance or autocorrelation functions when there is no need to distinguish them. 

In this paper we derive the sampling properties of correlograms for complex valued higher-order white 
noise with arbitrary mean. The specific correlograms we consider are the well known lag windowed au- 
tocorrelation estimator, and the lag windowed autocovariance estimator in which the sample mean is sub- 
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tracted from the data. To contrast with the ubiquitous lag windowed correlograms, we also consider the 
fixed-length summation autocorrelation estimator. 

The sampling properties of the estimators presented here are derived to second-order. This includes the 
bias, the covariance and the mean-square-error (MSE) and also the variance and consistency. 

2 Definitions and conventions 

In this section we introduce some basic definitions and notational conventions. 

In practice most correlograms are based on sequences of data samples Z[n] where neZ = {0, ±1, ±2, ...}. 
Here we denote the sequence by putting a square bracket around the independent variable to distinguish 
them from functions such as Z(t) t E R where the independent variable is continuous. The correlogram of 
sequences will also be a sequence so we favor the terms autocorrelation sequence (ACS), denoted R[l] and 
autocovariance sequence (ACVS), denoted C[l], over the often used terms autocorrelation function (ACF), 
denoted R(l) and autocovariance function (ACVF), denoted C(l). 

Since we will be dealing with statistical estimators we will need to distinguish between ensemble, or 
population, quantities and the estimates, or samples, of the quantities. We will put a hat accent on the 
symbol used for the estimators of a quantity so, for instance, R is an estimate of R. 

The distinction between the ACS and the ACVS is as follows. We define the ACS as 

Rzz[l] :=E{Z*[k]Z[k + l]}, l,k£Z 

while the ACVS is defined 

C ZZ [l] ■= Cov {Z[k}, Z[k + I}} :=F l {Z*[k}Z[k + l}}-E{Z*[k}}E{Z[k + l}} 

where E{-} is the expectation operator, Cov {•, •} is the covariance operator, I is the lag, and Z*[k] is the 
complex conjugate of Z[k\. Thus the difference between the ACS and ACVS is that, in the case of the 
ACVS, the mean of the sequence is explicitly subtracted out, or removed, while it is not removed in ACS. 
Note that C..,R.., and, Cov {•,•}, are all defined so that their first argument becomes complex conjugated, 
this so that the default product between two unconjugated arguments will be a hermitian form. 

The definition of the ACS varies in the literature with regards to the placement of the complex conjuga- 
tion. Here, it is defined in such a way that if Z[k) was the complex harmonic sequence exp(z//c), the phase 
of its corresponding ACS would increase with increasing lag, viz. exp(— ifk) exp(i/(fc + /)) = exp(i/i). 

3 Generalised white noise 

General white noise sequences, in the sense we will use here, are a less restrictive version of IID distributed 
samples in which the cross-moments of any two different samples in the sequence are zero in the lowest 
orders but may be non-zero at a higher order. This is different from an IID since all their cross-moments 
are zero. It is also slightly more general that higher-order white noise since we allow for a non-zero mean 
and we also allow for a broader structure of complex moments. The reason for considering these process is 
driven entirely by the sampling error formulas themselves: the generalised white noise is the least restrictive 
class of sequences with a flat ACVS. 

3.1 Defining conditions of fourth-order general white noise 

For the purposes of determining the second-order sampling properties of second-order lagged-moment 
estimates we need to know the statistical lagged-moments of the signal only to fourth-order. In what 
follows, we introduce a general white noise sequence, which we denote e[n], with the following defining 
properties: the first-order lagged moment is 

E{e[n]} = n (1) 

where [i is the mean. 
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The ACVS of e[n] is zero for all non-zero lags, 



C ee [l] = Cov{e'[/c],e'[fc + Z]} = Cov {e'*[k], e'*[k + l]j = E {e'*[fc]e'[fc + l}\ = o 2 5 Q . u fc,J eZ 

(2) 

where a 1 is the (hermitian) variance and is the Kronecker delta function. 8 a ,b is equal to one if a = b 
and zero otherwise, hence Sqj represents the zero lag, I = 0. Equation expresses an important property 
of white noise, that its autocovariance function is zero for all nonzero lags. It is precisely this property 
which can be exploited of to test if a sequence is independent or not. Unfortunately as we shall see, 
all autocovariance estimates will in practice be distributed around zero and this distribution is specified 
according to the estimators sampling properties. 

The ordinary ACVS for complex sequences is a hermitian bilinear form. Since we are dealing with 
possibly complex valued sequences, the full description of the second-order properties requires also another 
lagged bilinear form, namely the lagged quadratic form covariance 

Cov{e'*[fr],e [fc-H]} =E{e [fc]e [fc + Z]} =m 2 S 0il = s 2 exp(i6 2 )6 0il (3) 

where m 2 is what we will call the quadratic variance. We call the left-hand side of Q the quadratic ACVS 
in contrast to the hermitian ACVS of the left-hand side of Q. An important property of the quadratic 
ACVS is that it depends on whether the signal is analytic or not. It is identically zero for random stationary 
analytical signals |3|. The quadratic ACVF has, to the best of the authors knowledge, not been explicitly 
investigated in the literature. This is because it is usually argued that only analytical signals are of practical 
importance. This is, however, does not mean the quadratic ACVS is without interest. In terms of the 
second-order sampling properties derived here, the quadratic ACVF is precisely what distinguishes purely 
real (non-imaginary) signals from complex signals. For purely real signals the quadratic ACVS is not zero, 
it is equivalent to the hermitian ACVS. 
The third-order lagged moment is 

E{e'*[k]e'[k + l]e'[k'}} = K 3 5 kk ,6 i (4) 

where K3 is the third-order cumulant of the zero lag and is related to the skewness of distribution. Finally, 
the fourth-order lagged moments of the white noise is 

Cov |e'* [k]e [k + l],e* [k']e [k' + I 1 ] } 

= (^4 - v 4 )5kk>5 i5oi> + a A 5kk'5w{\ - 8oi5w) + \m 2 \ 2 8i,k>-k8-Li>(l - 5oi5oi>) 
=(/x 4 - 2d 4 - \m 2 \ 2 )5kk'SoiSoi> + v 4 Skk'Su> + \m 2 \ 2 5i,k>-kS-U' 

=K45kk>8oi5oi> + cr 4 Skk>Sw + s 4 5k>,k+iS-i,i> (5) 

where /141s the fourth-order central moment and«4 = //4 — 2a 4 — \m 2 ] \' is the fourth-order cumulant. 
Fourth-order moments are related to the kurtosis of the distribution. 

To summarise, we specify a noise sequence e[-] which is given by a minimum of 5 low order cumu- 
lants, namely, the mean, variance, complex variance, third- and fourth-order cumulants. This is the least 
restrictive white noise specification relevant to the second order sampling properties of correlograms. 

3.2 Examples of higher-order white noise signals 

To clarify the general white noise sequence e[-] defined in the previous section, we give now some explicit 
examples. The ultimate example of a white noise signal is of course the Gaussian process with no memory. 
This is because the central limit theorem shows that most processes under general conditions will tend to 
Gaussian, and the no-memory aspect implies that the signal is white. The first four cumulants of a complex 
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Gaussian random variable are 



M = fJ>g 
s 2 = 

K3 = 

/«4 = 



complex Gaussian white noise e ~ N(fi g ,a 



(6) 



Gaussian white noise is, however, not the only type of white noise. Other kinds of white noise are based 
on non-Gaussian distributions. Such situations come about in practice if, for example, the values of the 
physical quantity being measured are not continuous as assumed in the Gaussian case. Examples of this 
are Poissonian and Bernoullian white noise. 

A Poissonian white noise model is appropriate when the signal values are limited to non-negative 
integers as, for instance, when the signal is a series of independent count values. The first four cumulants 
of the real Poissonian distribution, Po(A), which specify the Poissonian white noise sequence, are 



/! = A 



^3 

K4 



= A, 02 = 
A 
A 



> real Poissonian white noise e ~ Po(A) 



(7) 



Notice how the third and fourth order cumulants K3 and K4 are nonzero in contrast to Gaussian white noise. 
The reality of the signal comes from the fact that s 2 — a 2 and 62 = 0. 

The Bernoullian white noise model is appropriate when the signal values are binary, for instance, a 
sequence of independent yes-no decisions. We take the range of a Bernoulli random variable to be {0, 1} 
and the probability of getting a 1 is p (so the probability of getting a is 1 — p). The first four cumulants 
of the Bernoulli distribution, Be(p), are 



H = p 

a 2 = -p 2 + p 

s 2 = a 2 , e 2 = Q 

K 3 = 2p 3 -3p 2 +p 

k 4 = -6p 4 + 12p 3 ~ 7p 2 



P 



real Bernoullian white noise e ~ Be(p) 



(8) 



Again, the cumulants of order greater than 2 are not zero as opposed to Gaussian white noise and the signal 
is real because the two second-order cumulants are equal. The estimation of ACF of one-bit sequences and 
their sampling properties was discussed in I T3*l . 

A feature of both the Poissonian and Bernoullian noise is that they inherently have non-zero means. 
Ultimately, for large enough mean values both the Poisson and Bernoulli distributions can be shown to 
tend to Gaussian. But for finite mean values their cumulants are clearly incompatible with their Gaussian 
counterparts. 



4 Sampling properties of various correlograms for general white noise 

There are many different types of correlograms. These can be subdivided, see 0, according to whether 
the correlograms 1) explicitly involve an attempt to remove the mean of the signal, 2) correlate two dis- 
tinct signals or the signal with itself, and 3) normalisation based on the data samples is applied. The first 
dichotomy can also be seen as whether the intention is to estimate correlation functions or covariance func- 
tions. The second dichotomy is the distinction between cross- and auto-correlation respectively. The last 
one includes correlation functions which are normalised by an estimate of the covariance of the sequence 
and are sometimes known as coherence functions. 

We will now look in detail at various correlograms which are either estimates of autocorrelation se- 
quences or of autocovariance sequences. In particular we will examine the classical correlogram, the fixed- 
length correlogram, and the classical correlogram in which the sample mean is subtracted. Most standard 
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treatments allow the correlograms to be windowed arbitrarily according to a lag window w[l]. We however 
use a slightly different quantity for weighting the correlogram lag estimates which we will simply call the 
weight function W[l] which includes all normalisation factors. It is related to the lag window, which for 
w[l] = 1 gives an unbiased correlogram estimate, as W[l] — w[l]/ (N — \l\). As we allow the signal to be 
complex we use accordingly complexified correlograms throughout. As for the signal to be analysed, we 
will assume that it consists of a finite number, N, of samples of general white noise, e[-], as defined in the 
previous section. 

Our goal in this section is to derive the sampling properties of these correlograms up to second-order, 
which fundamentally are the expectation and the covariance of the estimators. We also provide some related 
sampling properties, such as the bias, the mean square error (MSE), asymptotic MSE for large number of 
samples. 

4.1 Lag window autocorrelation estimator 

We start by considering the most fundamental and simplest of all correlograms, the lag window correlogram 
i?^ 1 "'; see 1101 . It was popularised by Blackman and Tukey Q. Although it was originally expressed for 
real signals, it is easily generalised to complex signals by requiring it to be a hermitian form. Thus, for 
non-negative lags, we define the complex lag windowed correlogram as 

N-l 

R ( z w) [l] :=W[l]Y^Z*[k]Z[k + l} l>0 (9) 

k=l 

and the negative lags are just the complex conjugate of the correlogram with the opposite (positive) sign 

£rt]:=(if%|])* ;<o (io) 

Usually R% w ^ is used as an autocovariance estimator. This implies that the signal being analysed, Z, is 
assumed to have a zero mean, i.e. E{Z} = 0. However, in what follows, we will not assume that the mean 
is zero, in other words, we see as an autocorrelation estimator. Whether the data samples have a 

non-zero mean intentionally or through an oversight is irrelevant. If the mean is not known apriori and one 
wishes to remove it the usual method is to estimate the mean and subtract it from the data before estimating 
the correlogram. In this paper we regard this process as a distinct correlogram and the most commonly 
used example, the lag windowed autocovariance estimator, will be treated in section B31 

4.1.1 Expectation and bias 

The expectation of R^ w ^ [I] is 

N-l 

E [B^» [I] } = W[l]J2 (E [e* [k]e [k + 1} } + fiE [e* [k] } + M *£ [e [k + 1} } + | M | 2 ) = 

k=l 
N-l 

= W[l] ( E { e '* W [k + l]\ + l±E U*\k\ } + n*E |e' [k + 1} V) + (N - l)W[l]\fi\ 2 = 
fc=i 

N—l 

= W[l]^2a 2 S 0l + (N- l)W[l]\(i\ 2 = 

k=l 

= (N-l)W{l](a 2 S 0l + \[i\ 2 ) (11) 

The bias is just the difference between the expectation and the true, population value. In this case it is 
therefore, for all I, 

Bias {r? w) [1}} = E {Ri lw) [l}\ - R e e[l] = 

= (N - l)W[l](a 2 d 0l + H 2 ) - (<j 2 S m + H 2 ) = 

= ((N-1)W{1}-1)(<j 2 6 01 + \v\ 2 ), (12) 
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that is, 



Biasji?^]} = {{N -l)W[l]-l)(a 2 S ol + \fji\ 2 ), I = —TV, ... — 1,0 + 1, TV. (13) 



4.1.2 Covariance 

The covariance of a correlogram is just the covariance of the correlogram estimate between any two lags, 
say I and I'. The derivation of the covariance is as follows. For IV > 0, 

H^cov{^) m ,^)[r]} = 

N-lN-l' 

=J2 E ( Cov {z'*[k]Z'[k + 1], z'*[k']Z' \k' + + 

k=l k' = l 

+ H 2 (Cov {z'*[k],z'*[k']\+ Cov {z'[k + I], Z'[k' + + 

+ Cov { |/x| 2 , H 2 } + Cov |V * \k]Z [k + 1} , [iZ* [k'] } + Cov [k] , Z'* [k'] Z [k' + I'} } + 

+ Cov | Z'* [k] Z' [k + 1} , /x* Z' [k' + I'} } + Cov {/z* Z' [k + 1} , z'* [k'] Z' [k' + l']\ + 
+Cov ^Z* [k] , /x* Z' [k' + 1'} } + Cov {V Z' [fc + Z] , ^Z'* [fc'] | ) 

N-\l\N-\l'\ 

= E E (Cov{e'*[fe]e'[fe + Z],e , *[fc']e'[fc' + Z']} + 
fe=i fc'=i 

+ |/x| 2 (Cov{e'*[/c],e'*[/c']} + Cov |e' [fc + Z],e'[fc' + /']}) + 
+ + [iE U[k\e*\k + l]e'*[k']\ + f i*E U[k}€*[k']e'[k' + 
+ fi*E |e'[fc]e'*[fc + l]e'[k' + I'}} + fiE {e'*[/c + l]e*[k']e [k' + 
+ (Ai*) 2 Cov |e'* [k],e [k 1 + Z'] } + ^ 2 Cov {e [k + I], e'*[k'\ }) 

N-\l\N-\l'\ 

= ^ E ~ 2o " 4 ~ l m 2| 2 )^fe"^o;<^o;' + v 4 Skk>$u> + s 4 6k,k>$o,iSa,i> + |/x| 2 (a 2 6 k k> + c 2 4+z, &'+/') + 

k=l k' = l 

+^nlS k k'Soi + ^*K 3 5k k >S i> + n 3 5 0l Sk-k'M + Ai«3<5o/'4'-fc,; + (v*) 2 m 2 5 k ,k>+i> + ^ 2 ra* 2 5 k ' ,k+i) 

N-\l\N-\l'\ 

= E E ((«4 + s 4 )4fc'^o;^o;' + {^l^kk' + M* K 34,fc'+;')<$o; + {^*^hk' + ^l^k' ,k+i)Sm> 
k=i fc'=i 

+cr 4 4fc'5«' + Im| 2 o" 2 (4fe' + 5fe+;,fe'+/') + {n*) 2 m 2 8k, k '+i' + ^ 2 m* 2 5 k > ,k+i) 
=6oi5 01 ,{k 4 + s 4 )N + 5 i(hk* 3 (N - I') + ^* K3 (N - I')) + <M/i*K 3 (iV - + - 0) + <W(TV - Z)+ 

+ |^|V (TV - max(Z, Z') + TV - max(Z, Z')) + {n*) 2 m 2 {N — min(Z + Z', TV)) + fi 2 m 2 (N — min(Z + Z', TV)) 
=5oj<M«4 + s 4 )TV + + M*K 3 )(iV - I') + (5 ;'O*«3 + hk* 3 ){N - Z) + <W 4 (TV - Z)+ 

+ 2\n\ 2 a 2 (N - max(Z, I')) + ((fi*) 2 m 2 + ^ 2 m* 2 )(N - min(Z + I', TV)) = 
=5qi5oi>(k 4 + s 4 )TV + 2%»(/*«S)(JV - J') + 2<y i'»(M«3)( J V - + ^/'^ 4 (TV - + 

+ 2|^| 2 cr 2 (TV - max(Z, /')) + 23?(/i 2 m^)(TV - min(/ + I', TV)) 

where we have used the summation formulas J42l >. i43\ . J44l l. ( 145 > . and fl46b . Here, 3?(-) is the real-part 
operator, min(a, b) is equal to the argument, either a or b, which is less than or equal to the other argument, 
and conversely max(a, b) is equal to the argument which is more than or equal to the other argument. 
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From the above derivation we have thus found that the covariance is 



w[l l w[n Cov {% lw) [t\M w) [l']} = *kMk4 + s A )N + 2S 0l ^ K * 3 )(N \l'\) + 26oiM(»k* 3 )(N - \l\)+ 
+ 5 u ><t\N - \l\) + 2| M | V (N - max(\l\, \l'\)) + 2^(p 2 m* 2 )(N - min(|/| + \l'\,N)), 

W > (14) 



For IV < 0, 
W ^Cov{^) W ,^) [ , ] } = 

N-IN-V 

= Y^ ( Cov { z '\ k ] z '*\ k + l],z'*[k']z'[k' + ;']} + (^*) 2 Cov {z'[k], z'*[k']} + m 2 Cov {z'*[k + l],z'[k' + V}} + 
k=l k'=l 

+ Cov { H 2 , | M | 2 } + Cov \z' \k\z'* [k + l], nZ'* [k'] } + Cov \ji*Z \k],z'* \k'\z' \k' + /']} + 

+ Cov { Z [k] Z * [k + 1} , Z \k' + I'} } + Cov { fiZ * [k + 1} , Z * \k'\ Z [k 1 + I'} } + Cov { ^* Z [k] , n* z'[k' + + 

+ Cov{nZ'*[k + l],iiZ'*[k 1 ]}) 

N-\l\N-\l'\ 

= zZJ2 ( Cov { e * W 6 ' t fc + *1 ' e '* M 6 ' + } + (M* ) 2 Cov |e' [k] , e'*[k'} } + M 2 Cov |e'* [fc + I], e [k' + V] } + 
fe=i fc'=i 

r /. ... .. / i _ r / / /.. . ...i 

+ 



+ + fiE je* [jfe]e' [fc + /]e'* [*'] } + y,E |e'* [fc]e'* [k']e [k' + /'] } 
+ n*E {e*[k]e [k + l]e \k' + I'} } + n*E |e' [fc + l]e* [k']e [k 1 + I'] } 
+ |m| 2 Cov [k],e Ik' + + |m| 2 Cov |e'* [fc + l],e* [k'}}) 



+ 



N-\l\N-\l'\ 

= Z^2z^2 ^ 4 ~ 2<ji ~ l m 2| 2 )4fc"5o/<W + s i S kk >Sii> + <y 4 S k .k'SouS M + {v*) 2 m* 2 8 k k> + ^ 2 m 2 S k+ i.k'+i' + 

k=l k' = l 

+ fiK^Skk'Soi + ^inlSkk'Soi' + n* n 3 5 i5k-k> ,i> + ^* K 3 S i'Sk'-k,i + 

+ \[l\ 2 <T 2 S k ,k'+l> + \ii\ 2 (J 2 5k>,k+i) 
N-\l\N-\l'\ 

= X! ((«4 + V 4 )8kk>SoiSoi> + (M k 3<W + ^* K 3^kM'+l')Soi + {^K 3 S k k' + M* K 34',fc+i)<W + s 4 5 kk/ 6 u > + 

k=l k' = l 

+ H 2 o" 2 {5k,k'+v + S k ',k+i) + (p*) 2 m 2 S k .k' + fJ 2 m 2 Sk+i,k'+i') 
=Soi5oi>{K4 + v 4 )N + S 0l (fin* 3 (N - V) + fi*K 3 {N - I')) + S 0l ,(^* 3 (N - I) + f i*n 3 (N - I)) + 8 u ,s 4 (N - l) + 
+ \y\ 2 G 2 (N - max(l, I') + N - max(Z, /')) + (n*) 2 m* 2 (N - min(Z + N)) + fi 2 m 2 (N - min(Z + I', N)) 

=SoiSoi>{K4 + <T 4 )N + <5 /(M«3 + M*«3)(^V - /') + <W(M*«3 + VK* 3 )(N -l) + 5u'S 4 {N - l) + 

+ 2\n\ 2 a 2 (N - max(Z, I')) + {{n*) 2 m* 2 + ^ 2 m 2 ){N - min(Z + I', N)) = 
=6 i5 i,{K4 + a 4 )N + 26 0l ®(p K *){N - V) + 2S 01 ^k* 3 ){N - I) + S w s 4 {N - l) + 
+ 2\v\ 2 cr 2 (N - max(Z, I')) + 23?(/i 2 m 2 )(iV - min(Z + V, N)) 

so 

WflWpf ™ {% lW) ^ R ( J w) in} =SoiSov(k 4 + a 4 )N + 2S ol n(pn* 3 )(N - \V\) + 2<5 ^( M «*)(iV - 

+ 5 U , S 4 (N - \l\) + 2| M | V (N - max(|/|, |/'|)) 

+ 2»( M 2 m 2 )(iV-min(|;| + |r|,iV)), W < (15) 
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4.1.3 Mean square error 



Now the mean square error (MSE) is the variance (I = I') plus the bias squared. The derivation of the MSE 
in this case starts from this definition and proceeds as follows: 

MSE = Var{fl^[Z]} + Bias {r^ [I] } 2 = 

=W 2 [l] (*oi((«4 + s 4 ) + 4St(ji/j%))N + (ct 4 + 2|/z| V) (A - I) + 2^ 2 m* 2 )(N - mm(2l, A))) + 

+ \(N~l)W[l]~l\ 2 (<j 2 5 0l + \ f ,\ 2 ) 2 = 
=W 2 [l] (Sqi{(ka + s 4 ) + 43t(jii4))N + (<r 4 + %| V) (N - I) + 2$R(,i 2 to*)(A - min(2Z, A))) + 

+ ((A - l) 2 W 2 [l] - 2(A - l)W[l) + l)((a 4 + 2a 2 | M | 2 )<5 (U + \n\ A ) = 
=S Q i (((«4 + s 4 ) + 45i( w 4))AW 2 [0] + ct 2 (ct 2 + 2|^| 2 )(A 2 W 2 [0] - 2AW[0] + 1)) + 

+ (ct 2 (ct 2 + 2H 2 ) (N-l) + 2^ 2 m* 2 )(N - min(2Z, A)) + |/i| 4 (A - I) 2 ) W 2 [l] - 2| M | 4 (A - l)W[l] + \n\ 4 
=S ol (((/S4 + s 4 ) + 43?(MMs) + *V + 2M 2 )A)AW 2 [0] - 2Act 2 (ct 2 + 2| M | 2 )W[0] + a 2 (a 2 + 2|/x| 2 ))) + 

+ (a 2 {a 2 + 2H 2 ) (N - 1) + 25R( A1 2 m*)(iV - min(2Z, A)) + H 4 (A - I) 2 ) W 2 [l] - 2|^| 4 (A - l)W[l] + |/i 

Thus we find that the MSE is 
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mse|e^[o]} = 


(k 4 + s 4 + 


<r 4 + 2| M |V 


+ 49%/c£) + 2%l(p 2 m* 2 ) + (a 2 - 


f M 2 ) 2 a) AW 2 [0]- 


h 




- 2N {a 2 ^ 


-\n\ 2 ) 2 w[o] 


f(- 2 + H 2 ) 2 




(16) 


MSE{flP^[l ^ 0]} = 


{a 2 (a 2 + 2 




f- 2$t(ii 2 m* 2 )(N - min(2Z, A)) + 


M 4 (a~/) 2 )w 2 [/]- 


h 




-2\^(N 


-l)W[l} + \fi 


4 




(17) 



Asymptotically, i.e. A — > oo, keeping only the leading terms of in AT or Z for each coefficient of each 
power of W[l], we find 

MSE {i?f^[0]} = (|/i| 2 + a 2 ) 2 N 2 W 2 [0] - 2A(ct 2 + \^\ 2 ) 2 W[0] + {a 2 + \^\ 2 ) 2 (18) 
MSE{^)^0]} = |/i| 4 (A - l) 2 W 2 [l] - 2\^(N - Z)W[Z] + M 4 (19) 
where we assume that /i ^ 0. 



4.2 Fixed-length summation autocorrelation estimator 

As an example of a correlogram that does not have form of the classic lag windowed correlogram, as given 
by equation we present what we will call the fixed-length summation correlogram denoted Rj . It is 
defined as 

L 

R { p[l]:=W[l]^Z*[k]Z[k + l] 0<1<N-L = M, (20) 
k=l 

This estimator is defined in |3|, equation (8.96). It is implemented on the DWP electron counts correlator 
experiment 1 6 1 onboard the CLUSTER-II space mission. Its defining characteristic is that the same, fixed 
number of terms are summed over for all lags. This in contrast with the classic correlogram for which the 
number of terms decrements with increasing lag. This implies that the lags of the classical correlogram 
all have different statistics, which is an unwanted property. With a fixed length summation however, one 
might hope that each of the estimated lags will have the same statistics giving more equitable lag estimates. 
As we will now see, this is in fact true but only when the mean is zero. 
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4.2.1 Expectation and bias 

The bias is for a general signal 



E [I] }=W[l\Y,E{Z*[k]Z[k + l]} 

k=l 
L 

= w[i\Y J Rz[i] = 



k=l 

L 



= W{l]R z [l]J2l = 

fe=i 

= LW[l]R z [l] 



The bias is then 



Biasjfl^Z]} = e{r { P\1]} -R z [l] = 

= LW[l]R z [l] - Rz[l] = 
= [LW[l] - l)R z [l] 

Thus with a window choice of W[l] = 1/L the fixed-length correlogram is unbiased irrespective of the 
signal. For the special case of the higher-order white noise signal, e, with which we are mainly interested 
in here, the bias is therefore 



Bias{#^[Z]} = (LW[l] - 1) (a 2 S .i + |m| 2 ) (21) 



4.2.2 Covariance 

The covariance of the fixed-length correlogram can be derived accordingly, 



' r Cov = 



W[l}W[l'} 

L L 

=1212 (Cov {z'*[k]Z [k + 1], z'*[k']Z [k' + 1'}} + 
fc=lfc'=l 

+ |^| 2 (Cov{z'*[k],z'*[k']} +Cov{z'[fc + l],z'[k' + /']}) + Cov{| M | 2 >| 2 } 
+ Cov \z* \k\Z [k + l], [iZ* \k'\ } + Cov {y,Z* [k] , Z* \k'\Z \k' + /']} + 
+ Cov { Z * [k] Z [k + 1} , n* Z [k' + I'} } + Cov { Z [k + 1} , Z * \k!\ Z \k! + I'} } -+ 
+Cov { nZ* [k] , Z \k' + I'} } + Cov { /j,* Z [k + l], fiZ'* [k'] } ) = 

=J2Y1 (Cov{e'*[fc]e> + Z],e'* [&']£>' + /']} + 

+ (ji* ) 2 Cov |e' [k] , e'*[k'\ } + ^ 2 Cov |e'* [k + 1} , e [k' + I'}} + 0+ 
+ iiE {e'*[k]e [k + l]e* [k'] | + \iE |e'* [k]e* [k'je [k' + I'} } + 
+ n*E |e'* [k]e [k + l]e \k' + I'} } + n*E |e' [k + l]e* [k']e [k 1 + /']} + 



L L 



fe=lfe' = l 
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+ |/i| 2 Cov y [k],e'[k' + + |/x| 2 Cov {e'*[k + I], e'*[fc']}) = 

L L 

=^~] ^ ((M4 - 2ct 4 - |m 2 | 2 )4fc"5o/<^o;' + s 4 5 k k>Su> + (T 4 5 k ,k>So,i6o,i' + {^*) 2 m 2 S k k' + p 2 m 2 5 k +iM'+v + 
k=ik'=i 

+HKl5kk'Soi + pulSkk'Soi' + p*K3SoiS k -k',i' + IA*Ka8oi>5k>-k,i + \lA a $k,k>+v + \^\ 2 cr 2 Sk',k+i) = 

L L 

^ ((«4 + s 4 )4fc'<5o/(^oi' + i^lSkk' + p*K 3 6kM'+i')Soi + (n*K 3 5kk> + fiK^Sk' ,k+l)Sov + 

k=lk' = l 

+<J 4 5 k k'Sw + Im| 2 c 2 (Sk,k'+l> + $k>,k+l) + (p*) 2 m 2 6 k ,k' + fi 2 m 2 S k +i,k' +v) = 
=SuiSqi>(k4 + s 4 )L + d i {^k* 3 L + p*K 3 (L - min(|/'|,L))) + 5 i> (h*k 3 L + fin* 3 (L - mm(\l\,L))) + 

+ 5 w a 4 L+\p\ 2 a 2 (L - mia(\l'\, L) + L - min(|Z|, L)) + {p*) 2 m 2 L + p 2 m* (L-mm(\\l\ - \l'\\,L)) = 
=5 i8 w {n 4 + s 4 )L + 6 0l (23?(^k*)L - y*K 3 mm(|Z'|, L)) + 5 i> (2^k* 3 )L - /ik* 3 min(\l\,L)) + 

+ 6 u ,a 4 L + H V [2L - min(|Z|,£) - min(|Z'|, L)) + 23ft(> 2 m|)Z - n 2 m* min(||Z| - |Z'||,Z) 

where we have used the summation formulas ( 147 > . d48l . ( I49> . (15 0> . and J5 II . 
So finally we can write the covariance for all I and I' as 



i^--Cov {r^ [I], [/'] } = 5 w *oi' («4 + s 4 )L + S ol (2$t(fiK* 3 )L ~ fi* K3 mm(\l'\, L)) + 



W[l}W[l'} 



+ 5 i> (2^(fiK* 3 )L - fiK* 3 min(|Z|,£)) + S w a 4 L+ 
+ \p\ 2 a 2 (2L-min(|/|,L) -mm(\l'\,L)) + 2^i(fi 2 m* 2 )L - fi 2 mlmin(\\l\ - \l'\\,L) (22) 



For convenience, we also provide the covariance for the case of real-valued variates 

—i—CovfflC/Opj^t/Op']} =5 0l S 0l ,( Ki + s 4 )L + 6 0lf MK 3 (2L - min(|Z'|, 

+ 5 m ,fiK 3 (2L - mm(\l\,L)) + 5\ l ^ V \<r 4 L+ 

+ n 2 a 2 (4L - min(|Z|, L) - min(|/'|,L) - min(||Z| - \V ||, L)) 

(23) 

and if in addition \l'\ < \l\ < L , then 

W[l]W[l'] C ° V [l] ' I = SwS ° 1 ' (K4 + sA)L + 6m2 ^ L ~ \ 1 ' I/ 2 ) + S oi' 2 ^( L - N/ 2 )+ 

+ ^ ( j 4 L + 4 M V(L- |Z|)/2) (24) 

The results shown above are expressed for both zero and nonzero lags. Formulas can also be given 
which are explicit in the zero and nonzero lags. They are as follows: 

Var {^[0]} = W 2 [0}L (k 4 + s 4 + <t 4 + 2|^| V + 2^ 2 m* 2 ) + 4»(/<«5)) (25) 

Vai^R^U ^ 0]} = W 2 [l] (La 4 + 2\p\ 2 u 2 (L - min(|Z|, L)) + 2m{p 2 m* 2 )) (26) 

W[0]W[l] COV {xi fl) l°l% fl) W} = 2L ( R 0«3) + 5R(M 2 ™S) + V + M*^3 + M 2 m^) min(|Z|,£), I 

(27) 

= |/a| 2 ct 2 (2Z/ - min(|Z|,i) - min(|Z'|,i)) 



+ 23fi(^ 2 m*)i-^ 2 m*min(||/| - |/'||,L), 1^0,1^0 
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4.2.3 Mean square error 

The mean square error of the fixed length correlogram for complex white noise with arbitrary mean can 
now be derived. The derivation proceeds from the variance and bias results derived above as follows 

MSe{#^[Z]} = Var{^ /z) [Z]} + iBias {^[Z]}| 2 = 
=W 2 [l] (M«4 + s 4 )L + 5 im(iJ,K* 3 )L + a 4 L + 2\^\ 2 a 2 (L - min(|Z|,L)) + 2%l{p 2 m* 2 )L) 

+ \LW[l] - 1| 2 (a 2 d i + H 2 ) 2 
=W 2 [l] (S iL(k 4 + s 4 + 4?R:{nK* a )) + o 4 L + 2|/x|V (L - min(|i|,L)) + 2ft(p 2 m*)L) 

+ (L 2 W 2 [l\ - 2LW[l] + 1) (a%t + 2a 2 \fi\ 2 Soi + |/i| 4 ) 
=801 (W 2 [l]L (k a + s 4 + 49f?Gu K *)) + ct 2 (<j 2 + 2\^\ 2 ){L 2 W 2 [l] - 2LW[l] + 1)) 

+ W 2 [l] (<r 4 L + 2\n\ 2 o 2 [L - min(|Z|, L)) + 25R(/?m*)L) + | M | 4 (L 2 W 2 [l] - 2LW[l] + l) 



-S ol (W 2 [l}L ( Ki + s 4 + m(fiK* 3 ) + La 2 (a 2 + 2| M | 2 )) + a 2 {a 2 + 2| M | 2 )(1 - 2LW[, 
+ W 2 [l] (a 4 L + 2|//|V (L - min(|/|, L)) + 2^ 2 m* 2 )L + L 2 \^\ 4 ) + \fi\ 4 (1 - 2LW[l]) 



Thus, the final formula for the MSE is 



mse{^ /;) [o]} 


=L (k 4 + s 4 + m(fM* 3 ) + L(a 2 + |^| 2 ) 2 + a 4 + 


2H 2 er 2 + 2ft([i 2 m* 2 )) W 2 [0] 




+ ((t 2 + |m| 2 ) 2 (1-2L>V[0]) 


(28) 


mse{£^[z ^ 0]} 


= (La 4 + 2|/a|V (L - min(|Z|,L)) + 2m{pL 2 m 


D + i 2 H 4 ) w 2 [/] 




+ |/i| 4 (1 - 2LW[l}) 


(29) 



Asymptotically as L — > oo and assuming // ^ the MSE becomes 

MSE{^[0]} = (a 2 + H 2 ) 2 (L 2 W 2 [0] - 2LW[0] + l) 

MSe{^)^0]} =\n\ 4 (L 2 W 2 [l] -2LW[1} + 1) 

where we have kept leading terms in L for each power of W. This proves that for the weight window choice 
W[l] = 1/L the fixed-length correlogram, as an autocorrelation estimate, is statistically consistent since 
the MSE is zero at all lags. If instead fi = then the asymptotic MSE of the nonzero lags is Lcr 4 |VV[Z] | 2 . 

4.3 Lag windowed autocovariance estimator using sample mean 

The two previous correlograms cannot be used as autocovariance estimators if the signal has an apriori 
unknown mean. As this is a common state of affairs one usually resorts to estimating the mean and sub- 
tracting this estimate from the signal. If the mean estimate is based on the same data samples as those on 
which the autocovariance estimate is to be based it is natural to see this scheme as a distinct estimator. In 
fact the mean need not even be explicitly calculated. There are a variety of autocovariance estimators, see 
191 . Here we only consider the simplest: the classic correlogram in which the ordinary sample mean is 
subtracted from the data samples. We denote it C^ w ^ and defined it for an arbitrary complex signal as 

C« w) [I] := W[l] £(Z fe - Z)(Z k+l - Z) (30) 

where 

_ 1 N 

Z =NZ2 Z * C3D 

fe=i 

is the sample mean. 
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4.3.1 Properties of the sample mean 

In order to derive the sampling properties 
estimator i31\ . These well know properties are that the sample mean estimator is unbiased, 



In order to derive the sampling properties of , we will need some of the properties of the sample mean 



f 1 N 1 1 N 1 N 

E { Z )= E \ j£> - ^H E {Zk} = *X> = " (32) 
I k=l ) k=l k=i 



that the standard deviation for independent samples goes is proportional to 1/yN, 

N N N N 

Var {e} = ±fl £ E {e\k]e[k'r} E {#] }' = £ (a 2 5 kk , + fi 2 ) — fj, 

k=ik'=i fe=ifc'=i 

2 N N 2 

= 

~~ AT 

where we have specialised to the generalised noise signal e[k]. 

4.3.2 Expectation and bias 

The expectation of C^' is 



2 



(33) 



E 



{Cgll]) = [N — l)W[l]a 2 (6 0l - l/N) (34) 



so the bias is 



Bias {C^[l}} = a 2 ((JVW[0] - 1)*,, - I^Ml) (35) 



4.3.3 Covariance 

We start by deriving an expression for the covariance of the correlogram for an arbitrary signal Z 

Wi ^cov{er» [i ],er>- [ r]} = 

N-lN-l' 

=Y^ Gov^z'[k]z'*[k + l}-Y*z'[k} - Yz'*[k + l] + [z 7 ] 2 , 

k=l k' = l 

z'*[k']z'[k' + i']-Yz'*[k']-Y* z'[k' + i'] + |F| 2 } = 

N-lN-l' 

=xm ( G ° v { z> w z * t fc + z * t fc '] z ' t fc/ + 1 '\ } - Gov { z> w z '* i k + i i ^z'*^'} | + 
fe=i fc'=i 

- Cov | z' [fc] z'* [fc + i] , Z 7 * Z' [k' + l'}} + Cov | Z' [k] Z* [k + 1} , \ Y\ 2 X - Cov j Z 7 * z' [fc] , z'* [fc'] z' [k! + /']} + 
+ Cov ^Z r *z'[k],Z r Z'*[k r \\ + Cov ^Z 7 * Z' [k},Y* Z' [k' + /']}- Cov jz^z' [k], jiT] 2 } + 

- Cov \Yz'*[k + 1} , Z* [k']Z [k' + + Cov \Yz'*[k + 1} , YZ* [k'\ I + Cov ^¥z*[k + 1},Y*Z' [k' + + 

- Cov \z/z * [k + 1] , | z 7 ! 2 1 + Cov 1 1 z 7 ! 2 , z* [k'\ z [k' + 1'] } - Cov 1 1 z 7 ! 2 , zV* [&'] I + 

-Cov {jZ 7 ! 2 , Z^Z'lfe' + /']}+ Cov {iz 7 ! 2 , iz 7 ! 2 }" 
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where Z' := Z — fi. 

The last expression contains 16 terms involving various covariances of a general signal Z. To progress 
further we specialise Z to be the higher-order white noise signal e. Working out each of the 16 covariance 
terms individually in the order the appear above for this case we have: first covariance term 

Cov |e'[fc]e'*[fc + l],e'*[k']e'[k' + /']} = £o/<W4fc' (/m - & 4 ) + (1 - doi)5 kk >S W (T 4 + (1 - 4i<W)4,fc'+j'4',fc+i| m 2| 

= £o/<W4fc'(M4 - 2cr 4 - \m 2 \ 2 ) + 5 kk ,5 w a 4 + 4,fc'+i'4',fc+j| TO 2| 2 = 
= 8oiSoi>8 kk > K4 + 8 kk >Swa 4 + 4',fc+z4; ! z'|m 2 | 2 

second covariance term 

Cov{e'[k}e'*[k + l],7e'*[k'}} = ^(S ol S kk> (fi 4 - a 4 ) + (1 - 6 0l )(6 kk ,a 4 + S k , , k+l \m 2 \ 2 )) = 

= -j^(5 iS kkl (n4 - 2cr 4 - \m 2 \ 2 ) + (Wcr 4 + <5;,fe/-fe|m 2 | 2 ) = 
= —(5 iS kk iK4 + 8 kk >a 4 + 5 k ^ k+ i\m 2 \ 2 ) 

third covariance term 

Cov je'[fc]e'*[fc + Z],e'Y[fc' + = -^(£oi4,fc'+;'(M4 - ^) + (1 - 5m){5 k +i, k > +1'^ + 5 k ^ +v \m 2 \ 2 )) = 

= jf(5oi<>k,k'+l'(lJ-4 - 2cr 4 - \m 2 \ 2 ) + 5 k+ i, k >+i'<T 4 + 5 ktk > +v \m 2 \ 2 ) = 
= jj{$oi&k,k' +V + 4+i,fe'+i'cr 4 + 5 ktk > +v \m 2 \ 2 ) 

fourth covariance term 

Cov [e[k]e*[k + I], |7| 2 } = -^(Soiipi - o 4 ) + (1 - 5 w )(a 4 + \m 2 \ 2 )) = 

= ^2 (*0J(M4 - 2a 4 - |m 2 | 2 ) + a 4 + |m 2 | 2 ) - 
= -^( S MK4 + cr 4 + |m 2 | 2 ) 



fifth covariance term 



Cov{e'*e[k},e'*[k']e[k' + /']} = ^(<W4fc'G«4 - O + (1 - <5w ^ 4 + 4,fc'+Hm 2 | 2 )) = 



-^-(4r4fc'(M4 - 2cr 4 - |m 2 | 2 ) + (4fc'cr 4 + 4,fc'+i'|^2| 2 )) 

-^■(4i'4fe'K4 + 4/t'cr 4 + 4,fc'+H m 2| 2 ) 



sixth covariance term 



Cov{e'7[fc],e'e'*[fc']} = ^(4^4 - <J 4 + (N - l)a 4 ) + (1 - 4fc')K| 2 ) = 



-^(4fe'(K4 + A r cr 4 ) + |m 2 | 2 ) 



seventh 



Cov{e'7[fc],e'7[A;' + /']} - -^{6 k ,k>Ml* - <? 4 + (N - l)|m 2 | 2 ) + (1 - 4,fc'+;V 



= ^ (&,k'+J'(M4 - 2(T 4 + (iV - l)|m 2 | 2 ) + a 4 ) 

= ^2(4,fe'+i'(K4 + A r |m 2 | 2 )+ C r 4 ) 
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eighth 

Cov{77 [fc], |7| 2 } = - a 4 ) + (N l)(a 4 + |m 2 | 2 )) = 



N 3 N 2 
ninth 



= ^((/m - 2a 4 - |m 2 | 2 ) + 7V(a 4 + |m 2 | 2 )) = 
k 4 cr 4 + |m 2 | 2 



Cov |e'e'*[fc + Z],e'*[fc']e'[ fc ' +/']} = j^{Sk',k+l^o,l'{l^4 - c 4 ) + (1 - $oi>Sk> ,k+i)(5k+i,k>+i>>? 4 + h> M+i\m 2 \ 2 )) 

= jj(<>k',k+l5o,l'(H4 - 2cr 4 - |m 2 | 2 ) + 4+;,fc'+('cr 4 + 4',fc+H m 2| 2 ) = 

= (£fc',k+*^0,J'«4 + fik+^k'+l'V 4 + 4',fc+(| m 2| 2 ) 



tenth 



Cov {e'e'*[fc + Z],e'e'*[fc']} = ^(fe, fe+ ,(^4 - <J 4 + (N - l)s 4 ) + (1 - 6 k , <k+l )a 4 ) = 

= -^(5 k >,k+l(^ - 2<r 4 + (N- l)s 4 ) + a 4 ) = 



-^2 (fa',k+l{K4 + iV|m 2 | 2 ) + cr 4 ) 



11-th 



Cav{e'7[A;],e'e'*[A;']} = ^(<5fefe'(^4 - a A + {N - l)a 4 ) + (1 - S kk ,)\m 2 \ 2 ) 
= -^(Skk'(p4 - a 4 - |m 2 | 2 + (N - l)a 4 ) + |m 2 | 2 ) = 



-^(4fe'(«4 + iVcr 4 ) + |m 2 | 2 ) 



12-th 



Cov{e'e'*[fc + l], |e'| 2 } = ^(( M4 - <r 4 ) + (JV - l)(a 4 + |m 2 | 2 )) = 

= ^ (0*4 - 2cr 4 - |m 2 | 2 ) + N(a 4 + |m 2 | 2 )) = 
k 4 a 4 + |m 2 | 2 



N 3 N 2 



13-th 



Cov {|e'| 2 ,e'*[fc']e'[fc' + /']} = j^{5 w ( } H - o 4 ) + (1 - 5 w ){a 2 + |m 2 | 2 )) = 

= ^(^(M4 - 2a 4 - |m 2 | 2 ) + (a 2 + |m 2 | 2 )) = 



-^(S l'K 4 + cr 2 + |m 2 | 2 ) 



14-th 



Cov {\e'\ 2 ,e'e*[k'}} = cr 4 ) + {N - l)(a 4 + |m 2 | 2 )) = 

= 2a 4 - |m 2 | 2 ) + N(a 4 + |m 2 | 2 )) 



k 4 cr 4 + |m 2 | 2 
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15-th 

Cov {|7| 2 ,77[fc' + l'}} = ^ (( M4 - <r 4 ) + (N - l)(a 4 + |m 2 | 2 )) 



N 3 

= i(( M 4 - 2a 4 - |m 2 | 2 ) + N{o± + |m 2 | 2 )) 
k 4 , cr 4 + Im2| 2 



TV 3 iV 2 

and finally the last term 

Cov {|7| 2 , |7| 2 } = e {|7| 4 } - e {|7| 2 } e {|7| 2 } = 

1 4 
= ^(^4 + N(N - l)(2a 4 + |m 2 | 2 )) - ^ 

= i(/x 4 + (N - l)(2a 4 + |m 2 | 2 ) - Na 4 ) = 
K 4 , cr 4 + |m 2 | 2 



AT3 TV 2 

for Z > 0, I' > 0. Adding up all the terms we can now collect all terms containing the lag dependent delta 
functions [<5 i<W, {$01 + <W)> <*M'> !]• The term containing <5 /<W is 

+£ /<W(«4 + s 4 )4,fc' 

the terms containing #oi an d <W are 
the terms containing 8ij/ is 

+Si < i'(a 4 8 k ,k') 

and all the other terms are 

- (^5k+Lk'+l ,(j4 + (Sk,k'+l> + 5k\k+l)s i + 25fc i fc/(T 4 + (5 fe ,fe'+C + <5fe',fc+i) s4 _ 2 4 <5fe,fc' + 

1 / —2 — 2 + 3 
= - jr ( (24, fc' + 28k+i,k'+i' - 28 k k' H ^ )c 4 + 

+ (2<5fc,fc'+;' + 2S k ^ k+t — 8k,k'+i' — 8 k >, k +i H ^ )« 4 — -j^{28k,k' + ^fc,fc'+i' + Sk', k +i ~ jj, 

= - f(2^fe+i,fe'+/' - -^)<r 4 + (5fc,fc'+i' + *fc',fc+i ~ ^) s4 ~ ^(25 k ,k' + S k ,k>+i> + 8 k >,k+i - jj)J 

Now we perform the double sum over all fc, fc' each of the lag dependent delta terms individually and make 
use of the summation formulas (I42> . ( I43K 1441 . ( 145 l l. and ( I46> . First we have a contribution to the zero lag 
variance, 801801', which becomes 

+8 i8 0l > (« 4 + s 4 )(7V - max(7, J')) 
then a contribution to the zero lag versus any lag covariance terms, Sqi and 8oi', which become 

-£ ((2AT - max(, - 1 - ? - ^f^V c, + (JW - in-fl, - « - V - ) 
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then a contribution to the any lag variance term, Sijt, which becomes 

+5 ia ,<t 4 {N -max(l,l')) 

while all other terms become 

- 1 (<*(* - W/, 0) - ^-Of-V + *<* - „dn(l + *)) - M ), 
-^(2(JV - max(Z, /')) + 2(iV - min(Z + l', N)) - 3 ^"^ ~° )) = 
= - i (V - 2 max(Z, /')+/ + /'- ^)a 4 + (N - 2 min(Z + Z', N)) + 1 + 1'- ^).s 4 + 
-^(iV - 2 max(Z, /') - 2 min(J + l', N) + 3(1 + /' - ^)) 
Finally, adding up the individual terms, we arrive at the final expression for the covariance. For IV > 0, 

it is 



r C0V 



w[i]W[v] - {^M'^PI} = *>.«*>,«'(*« + <V - ab.i«* (i - - (i - §) 

4 /l 2max(|Z|,|Z'|) + 2min(|Z| + |r|,AQ-3(|Z| + |/'|) IV \ 
+ 8 ul ,a (N-\l\)+K A ^- ^ 3— j 

_ (J i( l _ _ _ s i( 1 _ 2 min (i^i + n N ) - m - iv 



N N 2 ) \ N N 2 J ' 

W > 0. (36) 



These results are comparable to (D Theorem 8.2.6 for the case of real white noise. 
For IV < 0, the analogous expression for the covariance is 

W [i]W[Z'] Cov {^ lw) [l],Ci lw) [l'}} = <W(«4 + s 4 )N - S , lKi f 1 - §) - *o,f«4 (l - § 

4 /l 2 mflJ c(|i| > |H) + 2min(|/| + |H,JV)-3(|f| + |n) K' 
+ S wm s (N-\l\) + K4 ^— — 2 3^ 

ff4 / _ 2max(|;|,|;'|)-^-|n _ }V\ _ 4 A _ 2min(|i| + m,iV)-|i|-|n _ 

2V iV 2 / S V TV N 2 



W < 0. (37) 



4.3.4 Mean squared error 



From the bias and the covariances one can determine the mean square error. From the definition of the 
MSE we find that 

MSE{<5 e (ta) [Z]} = Varjci^]} + Bias \d[ lw) [/]} 2 = 

-WB (*.,, (fa + »V - 2« 4 ) + ^ - o + « 4 (1 - 2 ' + 2 °"^ JV) - 6 ' - 3^) + 



N 



«.am/'r // 4n»t o x 4,., , , f 1 2Z + 2min(2Z, N) — 61 n I 2 ' 

=W 2 [«] U 0|J ((«4 + S )^ ~ 2k 4) + <7 4 (iV ~ I ~ 1 + ^ ) + «4 ( ^ 3^ 
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_ >4 ^ _ _ ^ + ^ _ 2)ww2[Q] _ 2(w _ 1)w[0] + 1} Jw + ™ 

=S i \ ((k4 + s 4 )N - 2k 4 M - lj + CT 4 (/V - 2)A^) W 2 [l] - 2(N - 1)<7 4 W[0] + a 4 ^ + 



, , Ar , „ Z 2 \ /l 2Z + 2min(2Z,iV)-6/ n Z 2 

, . _ 2min(2Z,i\Q-2Z _ f_\ (N — Z) 2 cr 4 \ 
' I jV J\T 2 / AT 2 / 



thus 



MSE {C e (hu) [0]} = {a A {N - 1)JV + {k 4 + s 4 )7V - s 4 - 2k 4 + ^)W 2 [0] - 2(JV - l)a 4 W[0] + <r 4 (38) 



Asymptotically i.e. ./V — > oo, keeping only terms of order ./V or /we find 

MSE {c e (hu) [0]} = ct 4 7V 2 W 2 [0] - 2A^j 4 W[0] + ct 4 (40) 



MSE 



{C £ (to) [^ 0]} =a 4 (Ar-0>V 2 [/] (41) 



5 Conclusion 



We have derived the sampling properties up to second-order of the ACS and ACVS estimators W- lw \ 
R^ l > and C^ w ^ for a general white noise sequence e[ ]. An interesting result we have found is that the 
covariances of the correlograms in general have a lag dependence. This is despite the fact that the noise 
sequence, e[-] for which these covariances were derived, is not itself lag dependent. 

Further conclusions based on the results derived here will be given in a following paper |7|. 

A Summation formulas 

In deriving the second order sampling properties we have used the following summation formulas for the 
lag windowed correlograms (with and without mean removal) in sections |4~T1 and |4~3*1 

N-\l\N-\l'\ 

Y Y Sk > k ' = N - max(\l\,\l'\) 0<|/|<iV, < |Z'| < AT (42) 

fe=i k'=i 

N-\l\N-\V\ 

Y Y S k',k+\i\ = N-mm(\l\ + \l'\,N) < \l\ < N, < \l'\ < N (43) 

k=l k' = l 
N-\l\N-\l'\ 

Y Y h,k>+\l'\ =N-min(\l\ + \l'\,N) < \l\ < N, < \l'\ < N (44) 
fc=i fc'=i 

jv-|/|jv-|;'| 

Y Y 6 k +\i\,k>+\l'\ = N - max (\ l \>\ l '\) 0<\1\<N,0<\1'\<N (45) 
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JV-|i|JV-|i'| 

J2 J2 l = W-\l\)(N-\l'\) 0<\1\<N,0<\1'\<N (46) 

fc=l k'=l 

The following sums were used with the fixed-length summation estimator in section l4~2l 

L L 



J2J2 5k - k '= L 0<\1\<M,0<\1'\<M (47) 
fe=ife'=i 

L L 

Y,Y,5 k , tk+]ll =L-min(\l\,L) < \l\ < M, < |Z'| < M (48) 

k=lk' = l 
L L 

^2Y,8 ki „ +m =L-wmQl'\,L) 0<\1\<M,0<\1'\<M (49) 
k=ik'=i 

L L 

^2J2 6 k+\iUk'+\i'\=L-min(\\l\-\l'\\,L) < \l\ < M, < < M (50) 

k=lk> = \ 

L L 

Y1Y1 1 = l2 - \ l \ - M > - - M (51) 

fc=lfc'=l 

where M := N - L. 
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